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Abstract 

Persistence studies in science, technology, engineering, and math (STEM) fields indicate that 
the pipeline to degree attainment is “leaky” and underrepresented minorities are not persisting 
in the STEM fields. Those students who do not persist in the STEM fields either migrate to 
other fields of study or drop out of higher education altogether. Studies of STEM student 
attrition point to a student perception of faculty disconnection from students, calling this the 
“chilly climate” (Seymour & Hewitt, 1997). Engagement theory states, “...it is the 
individual’s integration into the academic and social systems of the college that most directly 
related to his continuance in that college” (Tinto, 1993). A “chilly climate” in the STEM 
fields could then reflect in measures of academic and social engagement. This study uses the 
Beginning Postsecondary Longitudinal Student Survey, 2004-2009 (BPS:04/09) (Cominole, 
Wheeless, Dudley, Franklin, & Wine, 2007) and logistic regression analyses to examine 
academic and social engagements’ impact on STEM field persistence in postsecondary 
education, net of individual and institutional factors. Analysis by ethnicity, initial major, and 
engagement demonstrate that underrepresented minorities have different engagement patterns, 
but these engagement behaviors do not contribute significantly to staying in the STEM fields. 

Keywords: Postsecondary, Engagement, Persistence, STEM Education, Underrepresented 
Minority Students 
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1. Introduction 

College completion rates for all fields are “stagnant” (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 
2009): with only 59 percent of freshmen starting in 2005 at 4-year institutions completing a 
baccalaureate in 6 years (Snyder & Dillow, 2013). In the fields of science, technology, 
engineering, and math (STEM), national and societal demand in these fields creates special 
interest in increasing STEM field persistence and attainment. While STEM degree 
completion is on the increase since the beginning of the 21 st century, the persistence gap for 
underrepresented minority students in STEM is still a concern (National Science Board, 
2007). The college retention literature is robust and frequently relates student engagement to 
postsecondary persistence but with few exceptions it does not addresses student engagement 
measures directly with regard to persistence in STEM fields (NSSE, 2010). 

Special attention has been focused on the reasons why women and underrepresented 
minorities are less likely to complete degrees when their initial degree goal was in a STEM 
field (National Science Board, 2007). This concept has been called the “STEM leaky 
pipeline”, and is studied extensively (Griffith, 2010; National Science Board, 2007; Soe & 
Yakura, 2008) yet these studies fail to examine student engagement as a mechanism of STEM 
field attrition (Flynn, 2012). 

This study uses nationally representative longitudinal survey data from the National Center 
for Education Statistics’ (NCES) Beginning Postsecondary Longitudinal Student Survey 
2004-2009 (BPS:04/09) to make unique contributions to the literature. First, it uses 
BPS:04/09 data to examine the frequent assertion that underrepresented minorities do not 
persist in the STEM fields. Second, BPS:04/09 data are used to examine differences in 
engagement behaviors by ethnicity for students who start in the STEM fields. Finally, this 
study asks the question: Do either academic and social engagement impact student 
persistence for initial STEM declared students, controlling for ethnicity, student-level and 
institution-level factors? Analysis of persistence in STEM fields, along with measures of 
engagement, allows the determination of engagement’s impact upon STEM persistence for 
underrepresented STEM students. 

2. Previous Research 

The Digest of Education Statistics 2013 finds only 59 percent of first-time full-time students 
entering 4-year institutions in 2005 attained a degree at that institution within 6 years (Snyder 
& Dillow, 2013). Attainment has been studied using the nationally representative National 
Educational Longitudinal Study (NELS) and the High School & Beyond (HS&B) (Aldeman, 
2006; Bowen, Chingos, & Me Pherson, 2009; Tinto, 1993). These studies have examined 
factors such as gender, ethnicity, and financial need and the resulting literature has 
contributed to both policy and practice. Unfortunately, few of these nationally representative 
studies addressing degree attainment address persistence in STEM fields with engagement 
variables within their analyses (Flynn, 2014). To properly frame the current research, the 
literatures of both the traditional STEM retention and engagement theories are necessary. 
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2.1 STEM Majors and Engagement 

Prior to the mid-1990s the general consensus was that STEM attrition was due to the 
difficulty of the content (Seymour & Hewitt, 1997). In the mid-1990s studies addressing 
engineering retention found otherwise. Seymour and Hewitt asserted that institutional 
features rather than student characteristics contribute significantly to STEM postsecondary 
attrition. Most prominent was that students leave the science fields due to their perception of 
a “chilly climate”. This “chilly climate” is the students’ subjective perception of the STEM 
faculty as unapproachable, indifferent, and intimidating. Seymour and Hewitt (1997) 
published their qualitative study of “science” students’ leaving the field using the acronym 
SEM defining science as inclusive of both mathematics and engineering students. The 
research now differentiates with studies in engineering and technology separate from studies 
that still use the amalgam STEM designation. For this study, STEM will be addressed as both 
an amalgam (STEM) and separated into the science/math subfield of STEM distinct from 
engineering/technology subfield. 

2.2 The Campus and Institutional Context 

Following Seymour and Hewitt (1997), Berger, Ramirez, and Lyon (2005) emphasize that 
“retention is also a campus-based phenomenon” (p. 2). Ohland, Sheppard, Lichtenstein, Eris, 
Chachra, and Layton (2008, p. 259) state that there “is significant institutional variation” and 
“assert a need to address persistence and engagement at the intuitional level and throughout 
higher education” in engineering. In effect, Berger et al. (2005) ascribe the ability for students 
to engage as a function of the institutions providing opportunity for students to engage. 

Both Doolen and Long (2007), and Daempfle (2004) further confirm the “Chilly Climate 
Hypothesis” in survey studies where STEM faculty members were identified as being 
“unapproachable, cold, unavailable, aloof, indifferent and intimidating” by STEM students. 
Daempfle (2004, p. 41) concludes, “students were generally interested in the sciences but 
were ‘turned off’ by the structure and climate of the classroom”. Doolen and Long’s (2007) 
single institution study validated Seymour and Hewitt’s (1997) “structural or cultural 
sources” as leading to STEM attrition. Not surprisingly, the perception of a chilly climate led 
to low levels of contact with faculty and attrition in STEM fields. These levels of personal 
contact with faculty can be directly equated with Tinto’s (1993) concept of academic 
engagement indicating that climate impacts level of academic engagement behavior, 
specifically student-faculty contact. 

2.3 The Engineering Student 

Daempfle (2004) also addressed the differing cognitive abilities of students and found that 
the engineering students who switched majors out of STEM were not cognitively different 
from the non-migrating students, as was previously thought. Switchers worked as hard and 
had similar college GPAs to the non-switchers. These switchers were also found to be same 
“kinds of people” as non-switchers, and found to be no less qualified to master the necessary 
technical concepts as non-switchers. The only quality on which STEM switchers and 
persisters differed was that the switchers evaluated their academic experiences as 
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unsatisfactory. Daemptie’s term “switching” is also called migration (the change from one 
major to another) and was extensively analyzed by Ohland et al. (2008), with results 
consistent with Daemp tie’s. 

Ohland et al. (2008) used two large-scale databases, the Multiple-Institution Database for 
Investigating Engineering Longitudinal Development (MINDFIELD) and the National 
Survey of Student Engagement (NSSE), to explore persistence, engagement, and migration in 
engineering programs. They found that “engineering differs from other majors most notably 
by a dearth of female students and a low rate of migration into the major”. Using both the 
NSSE and the MIDFIELD, Ohland et al. (2008, p. 261) find that “except for the low 
proportion of women, engineering students are demographically similar to other college 
students”. Note that similar to the NSSE, the MIDFIELD collects data from only a subset of 
institutions (n = 9) and should not be considered nationally representative. 

2.4 The Leaks of the Leaky Pipeline 

While many students enter college with a specific major, many change majors during the 
initial years. This migration between initial and final major varies by institutions, yet some 
authors state that this is “especially the case” (Griffith, 2010) for the STEM fields. Citing the 
National Science Board (2007), Griffith (2010, p. 911) states “women and minorities are even 
less likely to persist in a STEM field major than are male and non-minority students”. Griffith 
used two large national datasets, the National Education Longitudinal Survey, 1988 
(NELS:88, n = 5500 from 1070 sampled institutions) and the National Longitudinal Survey 
of Freshman (NLSF, 1999, n = 1820 from 85 institutions). Griffith concludes by stating that 
“the environment of the institution and the STEM field departments can have strong impacts 
on the major choice”. The leaky pipeline could multiply STEM attrition for underrepresented 
minority students because retention of minority students is a problem for all majors (Swail, 
Redd, & Perna, 2003). 

The key points from both the STEM and the engineering-specific literature are clear. The 
research indicates two primary factors leading to attrition in engineering: (1) a lack of student 
interaction with faculty resulting from a chilly climate and (2) an agreement that the students’ 
ability is not solely responsible for their decision to change majors from engineering. 

2.5 Student Engagement 

According to Tinto (1993), “given individual characteristics, prior experiences, and 
commitments, ... it is the individual’s integration into the academic and social systems of the 
college that most directly related to his continuance in that college”. Tinto (1973) delineates 
two distinct categories of engagement—academic engagement and social engagement. 
Academic engagement involves students actively engaging with the faculty, their advisors, 
and study groups while social engagement is the students active involvement with the social 
aspects of college—affiliated clubs, sports, or participation in campus arts, drama, or 
theatrical activities. 

Student academic engagement is now well established as a contributory factor for 
postsecondary student persistence and attainment (Astin, 1993; Braxton, Hirschy, & 
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McClendon, 2004; Flynn, 2014; Kuh, et al., 2008; Kuh, Kinzie, Buckley, Bridges, & Hayek, 
2006; Kuh & Pascarella, 2004; Tinto, 1993; Wolf-Wendel, Ward, & Kinzie, 2009). Another 
study found a difference in the levels of both academic and social engagement by field of 
study (Flynn, 2012). When mean standard scores of student engagement index scores were 
differentiated by field of study using the BPS:04/09, the STEM fields demonstrated the chilly 
climate (see Figure 1). 



Figure 1. Chilly engagement in STEM fields 

Note. First-Year engagement by initial declared major illustrates “Chilly Climate Flypothesis” 
using nationally representative data from the BPS:04/09 dataset (From Flynn, 2012). 


Shortcomings associated with the existing research on engagement have prompted several 
scholars to call for student engagement analyses that based on large, multi-institution datasets 
using student level data (Kuh, et ah, 2008; Pascarella & Terenzini, 2005). The current 
availability of the BPS:04/09 allows for a nexus of these literatures to address empirically 
whether engagement differentially impacts STEM persistence (Flynn, 2014). The positive 
impact of student engagement and the negative impact of the student perceptions of a chilly 
climate in STEM are resonant. The relationship between academic engagement and the chilly 
climate has not been studied in large-scale analyses. 

3. Purpose 

The purpose of this study is to quantitatively examine the relationship between ethnicity, 
engagement, and STEM persistence. Do student engagement behaviors help explain the 
persistence gap in the underrepresented minority students in STEM fields? The hypothesis is 
that academic engagement is a significant influencing factor contributing to the attrition of 
underrepresented minorities from the STEM fields. The three intermediate questions are: 

(1) Do the BPS:04/09 data support that underrepresented minority students leave STEM 
fields? 

(2) Does the BPS:04/09 demonstrate differential engagement for underrepresented minority 
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students in STEM fields? 

(3) Do the differing engagement behaviors contribute to STEM attrition of underrepresented 
minorities? 

4. Methods 

4.1 Data Source 

The National Center for Education Statistics’ (NCES) Beginning Postsecondary Longitudinal 
Student Survey 2004-2009 (BPS:04/09) is well suited to provide information about the 
patterns of educational attainment and persistence in field. The BPS 04/09 provides extensive 
institutional and student self-reported survey information from 2004, 2006, and 2009 about 
persistence and student behaviors among a nationally representative sample of 16,700 
students who first entered college in the Fall of 2003 (Cominole, Wheeless, Dudley, Franklin, 
& Wine, 2007). The analytic sample consists of students who began their studies in four-year 
baccalaureate-granting institutions in 2003, including public, private, not-for-profit, and 
for-profit institutions. The sample consists of 8700 individual students from 1350 different 
baccalaureate-granting institutions. Missing data identified in the institutional characteristics 
(specifically school size, residential status, and tuition) is addressed by mean substitution 
procedures for the 1030 subjects (12% of the sample) with missing data. (Note 1) All 
analyses are weighted using the BPS:04/09 “WTB000” weight so “the study respondents 
represent the target population” (Cominole et ah, 2004, p. 107), in effect correcting for 
sampling bias. From the resulting weighted sample of 8700 students in the persistence 
analysis, 81% were still enrolled in higher education during the 2006 survey administration. 
All sample counts have been rounded to the nearest 10 to comply with National Center for 
Education Statistics’ reporting policies. 

4.2 Question 1 Methods 

“Do the BPS:04/09 data support that underrepresented minority students leave STEM fields?” 
To explore this question, the analytic sample (n = 8700) of first-year starting students at 4-year 
institutions was categorized into six initial (2004) fields of study and seven (2006) fields of 
study. The BPS:04/09 majors of life science, physical science, and mathematics were 
combined into a single field of study labeled “Science/Math”. Engineering and 
computer/information sciences were combined into the field of “Engineering/Technology”. 
The BPS:04/09 majors of education, business/management, health, vocational/technical, and 
other technical/professional were combined into a single “Professional” field of study for this 
analysis. Although not a major per se, the BPS:04/09 also identified students who were 
undeclared or not in a degree program in both the 2004 and 2006 surveys. The 
undeclared/undecided, social/behavioral sciences, and humanities were carried into the 
analyses as the fields of “Undecided/Undeclared”, “Social Sciences”, and “Humanities”. For 
the 2006 comparisons, a seventh category was created to capture students who were no longer 
enrolled. Since these could either be stop-outs (students who were currently not enrolled but 
who would return to complete their degree) or drop-outs (college leavers who will not return), 
the 2006 field of study has a seventh field labeled “Out”. 


190 


www.macrothink.org/jei 



,11 Macrothink 
All Institute" 


Journal of Educational Issues 

ISSN 2377-2263 
2016, Vol. 2, No. 1 


Three cross-tabulations were created to answer Question 1. In the cross-tabulations the rows 
represent the field of study declared at survey administration of the first year (roughly April of 
the student’s freshman year, 2004). The columns represent the field of study declared at the 
first follow-up survey administration (2006). Each cell represents percentages of the 2004 
fields of study (row header) enrolled in the field of study in 2006 (column header). The first 
table illustrates field migration for all 4-year college STEM students as of the first follow-up 
survey (n = 1350). The second table is field migration for only Black/African-American STEM 
students (n = 170) and the third is for only Latino STEM students (n = 140) from 4-year 
colleges. 

To determine significance of migration by ethnicity for STEM majors, the fields of 
Science/Math and Engineering/Technology are collapsed into a single field and a categorical 
variable to identify STEM persistence with three conditions (persisted in STEM, persisted in 
non-STEM, and no longer enrolled/left higher education). A multinomial logistic regression on 
this new variable and all ethnicity categories analyzes the relative risk ratio of staying in the 
STEM fields by ethnicity. 

All statistical analyses were run within STATA version 11 in compliance with the National 
Center for Education Statistics restricted data license agreement with sample sizes and 
student counts all rounded to nearest 10. 

4.3 Question 2 Methods 

“Does the BPS:04/09 demonstrate differential engagement for underrepresented minority 
students in STEM fields?” To explore question two, the analytic sample (n = 8700) was filtered 
to include only students who initially declared a STEM field (n = 1350: Science/Math n = 600, 
Engineering/Technology n = 750) when they enrolled at a 4-year institution. Weighting was 
applied with the WTB000 weight to create an estimated population size of 258,250 students 
nationwide. 

4.3.1 Dependent Variables 

Two additive model regression analyses address (1) first-year academic engagement and (2) 
first-year social engagement. The first set of regression analyses addresses academic 
engagement as a function first of gender and ethnicity, then utilizes five additive models 
controlling for three additive sets of covariates (additional student-level controls, institutional 
level controls, STEM subfield [either Science/Math or Engineering/Technology]). 

In order to address engagement as theorized by Tinto and subsequent researchers, The 
analyses utilize the BPS’s Academic and Social Integration Index scales. The questions 
composite to the Integration indices were asked at the initial survey (2004) to all participants. 
The initial 2004 student self-reported data were collected beginning in April of the students’ 
freshman year and represents their first-year engagement behaviors. 

The academic engagement index represents a mean score of the four items measuring student 
academic engagement behaviors. The four behaviors representing academic engagement are: 
(1) meeting with faculty outside of class time; (2) meeting informally/socially with faculty; (3) 
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meeting with advisors; and (4) participating in study groups. Students could respond with 
“never”, “sometimes”, or “often” (coded 0, 1, or 2 respectively). The index score was created 
by summing the responses, each multiplied by 100, and then dividing the total by four to 
yield a score between 0 and 200 on the index. The mean academic engagement for STEM 
majors is 90.00 (SD = 41.75). 

The social engagement index represents a mean score of the three items measuring student 
social engagement behaviors. The three behaviors representing social engagement are: (1) 
attends or participates in campus arts, drama, music, or fine arts activities; (2) attends or 
participates in campus clubs or organizations; and (3) participates or attends campus varsity, 
intennural, or club sports activities. Again, students could respond with “never”, “sometimes”, 
or “often” (coded 0, 1, or 2 respectively). The index score was created by summing the 
responses, each multiplied by 100, then dividing the total by three to yield a score between 0 
and 200 on the index. The mean social engagement index for STEM majors is 64.68 (SD=51). 
The descriptive statistics for the STEM starters in 4-year institutions in 2003 are presented in 
Table 1. 
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Table 1. Descriptive statistics for initially declared STEM students at 4-year postsecondary 
institutions in BPS:04/09 entering fall 2003 



Min 

Max 

Mean 

SD 

n 

STEM Starter Overall Persistence 

0 

1 

0.86 

- 

1350 

2004 Academic Engagement Index 

0 

200 

90.00 

41.75 

1350 

2004 Social Engagement Index 

0 

200 

64.68 

52.00 

1350 

2004 Academic Engagement Index 5 * 

-2.17 

2.60 

-0.02 

1.00 

1350 

2004 Social Engagement Index 5 * 

-1.24 

2.54 

-0.02 

0.98 

1350 

Male 

0 

1 

0.65 

- 

1350 

Black 

0 

1 

0.12 

- 

1350 

Latino 

0 

1 

0.10 

- 

1350 

Asian 

0 

1 

0.09 

- 

1350 

Other 

0 

1 

0.02 

- 

1350 

White* 

0 

1 

0.75 

- 

1350 

High School Grade Point Average 

1.20 

4.00 

3.40 

0.45 

1350 

Age 

20 

20 

18.39 

0.67 

1350 

Family Income 

0 

501,350 

76,920 

57,170. 

1350 

Total Aid Received 

0 

46,250 

10,220 

9,170. 

1350 

Dependent 

0 

1 

0.98 

- 

1350 

Tuition 

490 

39,030 

10,790 

8,430 

1350 

Highly Selective 

0 

1 

0.36 

- 

1350 

Moderately Selective 

0 

1 

0.51 

- 

1350 

Small Institution* 

0 

1 

0.23 

- 

1350 

Medium Institution 

0 

1 

0.23 

- 

1350 

Large Institution 

0 

1 

0.54 

- 

1350 

Non-Residential 

0 

1 

0.32 

- 

1350 

Public Institution 

0 

1 

0.61 

- 

1350 


Note. **: Standardized values; *: Omitted/reference variable within logistic regressions. 


4.3.2 Independent Variables 
(1) Ethnicity and Gender Variables 

The first model regresses both the academic and social engagement indices on ethnicity and 
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gender. Gender is a dichotomous variable labeled “male” and coded “1” for male and “0” for 
female. Ethnicity is also coded into 5 dichotomous variables: White, Black/African-American, 
Latino, Asian, and Other. The Other category includes American Indians from recognized 
tribes, Hawaiian Islanders, and Pacific Islanders. For the analyses White was the reference 
category. Due to participants’ ability to choose more than one ethnic/racial category the 
descriptive analysis percentages sum to more than 100% (Note 2). 

(2) Co variates 

Two categories of covariates were included in both regression analyses to address individual 
and institutional factors. Their inclusion within the analyses of the individual-level variables 
followed the literature on persistence. Attainment studies by Titus found gender and 
socioeconomic status (SES) were related to persistence (Titus, 2004); thus, gender and family 
income, as a proxy for SES, were included as individual-level covariates in the current study. 
Kuh et al. (2006) assert that financial aid significantly impacts college persistence, total 
financial aid received is also included as one of the individual-level variables. Closely related 
to both the financial aid and SES issues are the students’ status as a dependent of their parents 
(Swail et al., 2003), so student dependent status was included as an additional dichotomous 
covariate in this analysis. High school grade point average (GPA) and age are also included 
as individual-level covariates to control for prior achievement. The regression uses standard 
scores of high school GPA, income and total financial aid. 

The literature also informs the institutional-level covariate inclusion. Following findings on 
the financial considerations and their impact upon persistence and attainment (Kuh et al., 
2006; Titus, 2004; Swail et al., 2003), the standardized annual tuition was included as a 
covariate within the institutional-level controls. While the debate over institutional selectivity 
continues (Kuh & Pascarella, 2004), institutional-level covariates addressing selectivity status 
were included as two separate dummy variables representing “highly selective” and 
“moderately selective” institutions, leaving less than moderately selective institutions as the 
reference variable. Kuh also considers institution size as a significant institutional attribute 
contributing to student success (2004). Schools with more than ten thousand students were 
coded as large schools. Non-residential (less than 25% of the students residing on campus) 
was also included as a dummy variable in the analysis with residential schools (more than 
25% residing on campus) as the reference category. 

(3) STEM Subfield Variables 

In order to address a finer-grained understanding of initial STEM subfield and STEM 
persistence, dichotomous variables were created from the original BPS:04/09 data. The 
engineering/technology dichotomous (0-1) variable included students who declared either 
engineering or computer/information science during the initial wave. The science/math 
dichotomous variable was students who initially declared life science, physical science, or 
math as their major during the initial wave. The reference category in the regression analyses 
was engineering/technology. 
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(4) Interaction Terms 

Interactions between the two STEM subfields (science/math and engineering/technology) and 
the two underrepresented ethnicities (Black/African-American and Latino) created four 
additional dichotomous variables. 

4.3.3 Question 2 Analyses 

The investigation consists of two separate additive-model logistic regression analyses. 
Regression analysis is like finding an equation for a curve—with the dependent variable 
being the “Y” and the various independent variables (primary and covariates) being multiple 
“Xs”. The regression analysis results show the unstandardized coefficients for each 
independent variable. Each regression analysis table included here starts with the left-most 
column of coefficients representing the model including only the primary independent 
variables of. Each successive column of coefficients represents another additive model with 
additional independent variables or covariates added to the original set of variables. The 
rightmost column represents the full-model analysis, inclusive of all variables of interest and 
their relative impact upon the dependent variable. To address question 2, the analysis requires 
two separate regression analyses, each with parallel additive models, to address if either type 
of engagement is statistically different for underrepresented minorities in STEM. The first set 
of analyses uses the academic engagement scores as the dependent variable. The second set 
of analyses addresses social engagement scores as the dependent variable. Covariates were 
the same for both analyses, and the methods and structures are directly parallel. 

4.3.4 Regression Models 

Each analysis uses multiple sets of independent variables (ethnicity and gender, student level 
covariates, institution characteristic covariates, STEM subfields, and interaction terms), and 
reported the unstandardized ordinary least squared (OLS) slope coefficients for the respective 
standardized engagement index scores. 

The first analysis addresses academic engagement. Model 1 regresses academic engagement 
index scores on gender and ethnicity. This examines the relationship solely between gender, 
ethnicity and the academic engagement index score. The second model adds individual level 
student controls to the analysis. The third model adds institutional controls. The fourth and 
final model adds the different STEM subfields (Science/Math differentiated from 
Engineering/Technology) and the interaction of both underrepresented minority status and 
STEM subfields to the analysis. The final model adds covariates by interacting student initial 
major with their STEM subfield to determine the effect of the combination of major and 
STEM subfield on academic engagement index score. The overall purpose of this analysis is 
to determine whether academic engagement index scores significantly differ for 
underrepresented minority students. 

The second analysis is the same as the above analysis with the dependent variable being 
social engagement index score. The structure of the models and the sample are identical. The 
overall purpose of this analysis is to determine whether social engagement index scores 
significantly differ for underrepresented minority students. 
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4.4 Question 3 Methods 

“Do the differing engagement behaviors contribute to STEM attrition of underrepresented 
minorities?” To explore question three, the analytic sample (n = 8700) is again filtered to 
include only students initially declared in a STEM field (n = 1350: Science/Math n = 600, 
Engineering/Technology n = 750) and enrolled at a 4-year institution. All variable construction 
for question 3 are identical to the variable construction described for question 2 (see Section 
4.3.2). 

To determine differences in of STEM migration and attrition by ethnicity for STEM majors as 
a function of engagement, a multinomial logistic regression on categorical STEM persistence 
was performed. The engagement index scores and all ethnicity categories were included to 
show the relative risk ratio of staying in the STEM fields by ethnicity and engagement. The 
fields of science/math and engineering/technology are collapsed, and the dependent categorical 
variable identifies STEM persistence with three conditions (stayed enrolled in STEM [STEM 
Stayer], stayed enrolled in a non-STEM field [STEM Switcher], and no longer enrolled/left 
higher education [Out]) are used in the multinomial logistic analysis. Two separate 
multinomial logistic analyses are included, the first solely addressing engagement index scores, 
gender and ethnicity, and the second adding the student-level and institutional-level covariates, 
and separate subfields to capture science/math fields separate from engineering/technology 
fields. 

5. Findings 

5.1 Question 1 Findings 

The BPS:04/09 data support that underrepresented students leave STEM fields at a higher rate 
than do White, Asian, or students from other ethnic groups. Initial analyses for all students 
show that students migrate between fields of study between their first and third years. Of the 
students with declared majors, the range of in-field retention was from 53% (professional fields) 
to 43% (science/math and humanities). 


Table 2. STEM migration crosstabulation for all students at 4-year postsecondary institutions 
in BPS:04/09 entering fall 2003 


Declared Fields 

Science/Math 

2006 

Engineering/Technology 

2006 

Out (left school) 

2006 

Science/Math 2003 

43% 

2% 

10% 

Engineering/Technology 2003 

6% 

44% 

16% 


Table 2 shows migration for the students in the STEM fields (n = 1350): 43% of students 
remained in the science/math fields and 44% remained in the engineering/technology fields. 
Science/math shows a 1% increase in enrollment from 2003 by 2006 and 
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engineering/technology shows a 3% decrease in enrollment. Of the whole sample, 10% of the 
initial science/math starters and 16% of engineering/technology starters were no longer 
enrolled in school two years following their initial enrollment. 


Table 3. STEM migration crosstabulation for Black/African American students at 4-year 
postsecondary institutions in BPS:04/09 entering fall 2003 


Declared Fields 

Science/Math 

2006 

Engineering/Technology 

2006 

Out (left school) 

2006 

Science/Math 2003 

36% 

0% 

16% 

Engineering/Technology 2003 

4% 

35% 

22% 


Table 3 shows the sample of Black/African-American students (n = 190): 36% of the 
science/math starters and 35% of the engineering/technology starters were still in their 
respective fields two years following initial enrollment. Black/African-American STEM 
students showed a 1% decrease in science/math enrollment and a 5% decrease 
engineering/technology enrollment by their third year. Of the Black STEM starters, 16% of the 
science/math starters and 22% of the engineering/technology starters had dropped out of 
school within two years. 


Table 4. STEM migration crosstabulation for Latino students at 4-year postsecondary 
institutions in BPS:04/09 entering fall 2003 


Declared Fields 

Science/Math 

2006 

Engineering/Technology 

2006 

Out (left school) 

2006 

Science/Math 2003 

37% 

2% 

10% 

Engineering/Technology 2003 

2% 

37% 

17% 


Table 4 shows the sample of Latino students (n = 140): 37% of both the science/math starters 
and engineering/technology starters were still in their respective fields two years following 
initial enrollment. Latino STEM students showed a 1% decrease in science/math enrollment 
and a 5% decrease engineering/technology enrollment by their third year. Of the Latino STEM 
starters, 10% of the science/math starters and 17% of the engineering/technology starters had 
dropped out of school within two years. 


197 


www.macrothink.org/jei 















II Macrothink 
All Institute 


Journal of Educational Issues 

ISSN 2377-2263 
2016, Vol. 2, No. 1 


Table 5. Net change in STEM subfield percent declared enrollment comparing all students, 
Black/African-American, and Latino students 



Science/Math 2006 to 2003 

Engineering/Technology 2006 to 2003 

All Students 

1% 

-3% 

B lack/Afric an-American 

-1%* 

-5%* 

Latino 

i 

CO 

v? 

o x 

+ 

-5% + 


Note. + :p < 0.10; :/?<0.05. 


Table 5 summarizes the net change in the STEM subfields, comparing the change in subfield 
enrollment between 2003 and 2006 for all students and separately for Black/African-American 
and Latino students. 

The multinomial logistic regression results (see Table 6) found underrepresented minority 
students were significantly less likely than White students to remain in a STEM field. 
Compared to White STEM starters, remaining enrolled but in a field other than STEM was 
significantly more likely for Black students (Multinomial Logistic Relative Risk Ratio = 1.63, 
p < .05). Stated simply, the likelihood of switching is 1.63 times greater than that of staying in 
STEM. Compared with White STEM starters, dropping out of school entirely from STEM was 
also significantly more likely for Black students (Multinomial Logistic Relative Risk Ratio = 
1.95, p<. 05). 


Table 6. Risk coefficients comparing STEM switchers and school leavers to STEM stayers by 
ethnicity and gender 



STEM switchers 

School leavers from STEM 

Male' 

0.95 

2.04** 

Black/African- American 1 

1.63* 

1.95* 

Latino 1 

1.51 + 

0.92 

Asian 1 

0.56* 

0.30** 

Other Ethnicity (non-White) 1 

0.60 

-0.38 


Note. Observations n = 1350; + : y? < 0.10; : p < 0.05; ; p < 0.01; Dummy/dichotomous 0-1 

variable; Weighted multinomial logistic regression, relative risk ratios reported. 
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Compared with White STEM starters, STEM switching was nearly significant and more likely 
for Latino students (Multinomial Logistic Relative Risk Ratio = 1.51, p < .10). Stated simply 
the likelihood for Latino STEM students for switching is 1.51 times more likely. No significant 
difference with White STEM students was found for Latino STEM dropout rates. These 
migrations by ethnicity data are shown in Table 7. 


Table 7. STEM migration by ethnicity at first follow-up survey (2006) 



Black 

Latino 

White r 

Asian 

School Leavers (Outs) 

21%’* 

16% 

14% 

8% 

STEM Switchers 

41%’* 

44% + 

36% 

30% 

STEM Stayers 

38% 

40% 

50% 

62% 

n 

100% 

100% 

100% 

100% 


160 

140 

900 

120 


Note. Referenced category; + : p < 0.10, *: p < 0.05, **: p < 0.01; Significance difference from 
referenced category. 


The probability superscripts in this table are representative of the multinomial logistic 
regression significance levels relative to the referenced category of White STEM stayers. The 
difference between the ethnic minority groups is dramatic in Figure 2: Black students are 
both switching out of STEM and dropping out of college from STEM at a significantly higher 
rate than are White students. Latino students are switching out of STEM at nearly a 
significant level. 
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Figure 2. STEM migration percentages by ethnicity 
Note. STEM migration by ethnicity at BPS first follow-up survey (2006). 
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In answer to question 1, the analysis shows that unrepresented minorities are switching and 
leaving school from STEM at a higher rate than other ethnically identified students. This 
analysis, using the BPS:04/09, supports the leaky pipeline theory for minority students. 

5.2 Question 2 Findings 

The BPS:04/09 data show that underrepresented minorities have different levels of 
self-reported academic and social engagement than White and other ethnically identified 
students. 

Two separate additive-model ordinary least squared (OLS) logistic analyses with similar 
modeling using the nationally representative data yield engagement findings. All tables 
present results of the regressions as unstandardized OLS slope coefficients. In reporting 
unstandardized coefficients it is important to remember that a coefficient of 0 is the neutral 
position—equal likelihood/no increased or decreased likelihood over the reference category 
of White students. A positive coefficient indicates an increased or positive impact of the 
corresponding independent variable on the dependent variable. Negative coefficients indicate 
a negative impact or detrimental impact of the corresponding independent variable on the 
dependent variable. 
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5.2.1 Findings of Academic Engagement Regression Analyses for STEM Fields 


Table 8. Ordinary least squared slope coefficients predicting academic engagement index 
score as a function of ethnicity for STEM majors at 4-year institutions 



Gender & 
Ethnicity 

+ Student 

+ Institutional 

+ SM or ET and 

inter-actions 

Male 1 ' 

** 

-0.21 

* 

-0.19 

* 

-0.18 

-0.05 

Black/Afric an-Americ an 1 

0.26 + 

** 

0.37 

** 

0.39 

0.21 

Latino f 

-0.16 

-0.03 

0.07 

0.19 

Asian' 

-0.13 

-0.14 

-0.14 

-0.15 

Other (non-White) Ethnicity ! 

0.13 

0.16 

0.16 

0.11 

Age 


0.02 

0.06 

0.06 

HS GPA § 


0.07 

0.04 

0.02 

Family Income* 


0.05 + 

0.01 

0.01 

Financial Aid Received* 


0.13 

0.09 + 

* 

0.09 

Dependent 1 


0.65** 

0.54** 

0.56** 

Tuition 



* 

0.12 

* 

0.11 

High Selectivity 1 



*** 

0.42 

** 

0.38 

Moderate selectivity' 



** 

0.32 

* 

0.26 

Medium size' 



* 

-0.27 

-0.26 + 

Large size' 



-0.36** 

-0.33** 

Non-residential' 



-0.07 

-0.08 

Public 1 



0.18 

0.16 

Science/Math declared 1 




** 

0.27 

Engineering/Tech declared' 




0.00 

Science/Math x Black 




0.45* 

Science x Latino 




-0.25 

Engineering/Tech x Black 




0.00 

Engineering/Tech x Latino 




0.00 

Constant 

0.03 

-1.08 

-1.72 

-1.98 + 

Observations 

1350 

1350 

1350 

1350 

Pseudo R 2 

0.02 

0.06 

0.10 

0.12 


Note. + : p < 0.10; *: p < 0.05; p < 0.01; *: p < 0.001; Standardized values 

Dummy/dichotomous 0-1 variable; (MS) mean substitution applied as per text; 
Unstandardized OLS slope coefficients reported. 
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The academic engagement for STEM fields analysis is reported in Table 8. The first model 
demonstrates that compared with White, female students, being male is a negative predictor 
of academic engagement (B = -0.21, p < 0.01). In simple terms, male STEM students score 
significantly lower than do females on academic engagement. In model 2, we see being males 
still predicts lower academic engagement with the inclusion of student-level covariates in the 
analysis. With controls for individual covariates, being a Black/African-American STEM 
student is positively predictive of academic engagement (B = 0.37, p < 0.01). Also in model 2 
we see the significance of academic engagement score with income, amount of financial aid 
received and dependent status. In model 3, following the addition of institutional-level 
covariates, we see similar values for both male and Black/African-American students as from 
the previous model. The expected and previously documented significance attributed to 
institutional selectivity, tuition, and institution size is supported in the BPS:04/09 data. 

Model 4 includes the interactions between STEM subfield and interactions between both 
Black/African-American and Latino with STEM subfield. This analysis now shows no main 
effect for ethnicity in contrast to the previous models. Being a declared science/math major 
now predicts academic engagement relative to being an engineering/technology major: the 
science/math STEM subfield students academically engage more than do the 
engineering/technology subfield students (B = 0.27, p < 0.01). When subfields and ethnicity 
were interacted, compared with engineering/technology students, Black/African-American 
science/math students academically engage the most (B = 0.45, p < 0.05). With the effect size 
and significance of the institutional covariates dropping when subfields and interactions are 
included in the analysis, the indication is that it the discipline, not just the campus/institution, 
that impacts academic engagement. 

5.2.2 Findings of Social Engagement Regression Analyses for STEM Fields. 

The social engagement for STEM fields analysis is reported in Table 9. The first model 
demonstrates that for students, being male is a negative predictor of social engagement (B = 
-0.17, p < 0.05). Male STEM students also score significantly lower than do White females 
on social engagement. While both Black and Latino STEM students showed significant and 
negative coefficients for social engagement (B = -0.30, p < .001 and B = -0.29, p < .001 
respectively) in the initial model, the significance of the effects disappear with the addition of 
student- and institutional-level covariates. In model 2 we see that males still yield lower 
coefficients with the addition of student-level covariates. Now, with controls for student-level 
covariates, Black and Latino STEM students’ effect size drops and no longer demonstrate 
significance for social engagement. 
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Table 9. Ordinary least squared slope coefficients predicting social engagement as a function 
of ethnicity and gender for STEM majors at 4-year institutions 



Gender & 

Ethnicity 

+ Student 

+ Institutional 

+ SM or ET and 

inter-actions 

Male 1 

-0.17* 

-0.12 + 

-0.11 + 

-0.04 

Black/African-American 1 

-0.30*** 

-0.15 

-0.11 

-0.22* 

Latino 1 

-0.29* 

-0.16 

-0.07 

0.02 

Asian 1 

-0.21 + 

-0.22 + 

-0.21 + 

-0.22 + 

Other Ethnicity (Non-White) 1 

-0.02 

0.03 

0.02 

-0.00 

Age 


-0.05 

-0.03 

-0.02 

HS GPA § 


** 

0.10 

* 

0.08 

0.07 + 

Family Income 5 ’ 


0.12*** 

* 

0.08 

* 

0.08 

Financial Aid Received* 


0.11*** 

0.05 

0.05 

Dependent 1 


0.27 

0.18 

0.20 

Tuition 



** 

0.16 

** 

0.16 

High Selectivity 1 



* 

0.30 

* 

0.28 

Moderate Selectivity' 



* 

0.28 

* 

0.24 

Medium Size 1 



-0.24* 

-0.23* 

Large Size 1 



-0.28* 

-0.26* 

Non-residential 1 



-0.04 

-0.05 

Public r 



0.14 

0.13 

Science/Math declared 




* 

0.15 

Engineering/Tech declared 




0.00 

Science/Math x Black 




0.28 

Science/Math x Latino 




-0.21 

Engineering/Tech x Black 




0.00 

Engineering/Tech x Latino 




0.00 

Observations 

1350 

1350 

1350 

1350 

Pseudo R 1 

0.02 

0.07 

0.10 

0.11 


Note. + : p < 0.10, *: p < 0.05, **: p < 0.01, : p < 0.001; § : Standardized values; t 

Dummy/dichotomous 0-1 variable; Unstandardized OLS slope coefficients reported. 
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As with the academic engagement analysis, model 2 of the social engagement analysis 
illustrates the significance of income, amount of financial aid received and high school GPA, 
the covariates that have been studied elsewhere and were expected to relate strongly to 
persistence are also related to social engagement. In model 3, following the addition of 
institutional-level covariates, we see similar but slightly less negative slope coefficients for 
both male and Black students as compared with the previous model in this analysis. Here, too, 
we see the expected and previously demonstrated significance attributed to institutional 
selectivity, tuition, and institution size. 

Model 4 includes the interactions between STEM subfield and interactions between both 
Black and Latino with STEM subfield. This social engagement analysis shows only a 
negative main effect for ethnicity of Black STEM students (B = -0.22, p < 0.05). Being a 
declared science/math major predicts higher levels of social engagement relative to being an 
engineering/technology major (smaller effect size than with academic engagement, but 
equally significant, B = 0.15, p < 0.05). In other words, the science/math STEM subfield 
students socially engage more than do the engineering/technology subfield students (B = 0.27, 
p < 0.01). When subfields and ethnicity showed no significant interaction. 

In sum, there is a difference in the academic engagement of underrepresented minority 
students relative to White students. However, being Black in the science/math STEM subfield 
is a significant and positive predictor of academic engagement. Being Black is a significant 
and negative predictor of social engagement for these students. As for subfield, being a 
science/math student is positively and significantly predictive of both academic and social 
engagement (B = 0.27, p < 0.01 and B = 0.15, p < 0.05, respectively). 

5.3 Question 3 Findings 

The analysis of the BPS:04/09 data does not support that racially underrepresented students 
leave STEM fields at a higher rate than do White, Asian, or other racial/ethnic minorities as a 
function of engagement behaviors. 
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Table 10. Risk coefficients comparing STEM switchers and STEM school leavers as a 
function of ethnicity and gender compared with STEM stayers 



Ethnic itv 

+ Engagements 


STEM 

Switchers 

School 

Leavers (Outs) 

STEM 

Switchers 

School 

Leavers (Outs) 

Black/African-American 1 

1.63* 

1.95* 

1.65* 

1.93* 

Latino ! 

1.51 + 

0.92 

1.44 

0.83* 

Asian 1 

0.56* 

0.30** 

0.53* 

0.27** 

Other Ethnicity (non-White) 1 

0.60 

-0.38 

0.46 

0.69 

Academic Engagement 5 



0.88 

0.79 + 

Social Engagement 5 



0.92 

0.77 + 

Observations 1350 






Note. + : p < 0.10; *: p < 0.05; Dummy/Dichotomous 0-1 variable; § : Standardized values; 
Weighted multinomial logistic regressions, relative risk ratios reported. 


The multinomial logistic regression found nearly identical results as the initial comparison of 
STEM retention, switching, and postsecondary attrition (represented in the first two columns 
of Table 10), following the inclusion of the engagement variables in the multinomial regression 
(represented as the third and fourth columns). Table 10 shows that neither academic nor social 
engagement scores contribute a significant increased likelihood for switching out of STEM and 
only a trend (p <0.10) for reducing likelihood for leaving school when controls for gender and 
ethnicity are included in the analysis. After controlling for engagement, Black and Latino 
STEM students still have a significantly increased risk ratio for leaving school altogether (RRR 
= 1.90, p < 0.01; RRR= 0.83, p < 0.05 respectively). 
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Table 11. Risk coefficients comparing STEM switchers and STEM school leavers as a 
function of ethnicity and gender compared to STEM stayers inclusive of individual, 
institutional and STEM subfield covariates 



STEM Switchers 

School Leavers (Outs) 

Academic Engagement 5 

0.89 

0.83 

Social Engagement 5 

0.99 

0.92 

Male 1 

0.90 

1.84* 

Black/African-American 1 

1.14 

0.80 

Latino' 

1.20 

0.47* 

Asian 1 

0.64 + 

0.31* 

Other Ethnicity (non-White) 1 

0.35 

0.61 

Age 

1.06 

0.98 

HS Grade Point Average 5 

0.67*" 

0.56*** 

Family Income 5 

0.94 

0.61*** 

Financial Aid Received 5 

0.80* 

0.66" 

Dependent 1 

0.67 

1.07 

Tuition 

0.74* 

0.80 

High Selectivity 1 

0.86 

0.41* 

Moderate Selectivity 1 

1.17 

0.36" 

Medium size 1 

1.83** 

0.96 

Large size 1 

1.30 

1.02 

Non-residential 1 

0.74 + 

0.72 

Public 1 

0.68 + 

0.53 + 

Science/math declared major 2003 1 

1.47* 

1.24 

Engineering/tech declared major 2003 1 

0.00 

0.00 

Observations 1350 




Note. + : p < 0.10; *: p < 0.05; **: p < 0.01; **p < 0.001; § : Standardized values; ' 
Dummy/dichotomous 0-1 variable; Relative Risk Ratios reported. 


Only Black STEM students demonstrated a significantly higher risk of switching out of STEM 
(RRR = 1.63, p < 0.05) than White STEM students. Interaction of the engagement scores and 
the dichotomous ethnicity variables showed no significant interaction and no change in the 
effect of the main variables. 
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Table 11 includes student- and institutional-level covariates, and STEM subfields for a finer 
grained analysis. The addition of the individual and institutional controls, the analysis still 
shows that male STEM students are more likely to leave both school and STEM, Latino STEM 
students are less likely to leave school, and that between the STEM subfields, science/math 
students are the more likely to both switch out of STEM and to leave school altogether. 

6. Discussion and Conclusion 

Descriptive and multinomial logistic regressions confirm the migration of underrepresented 
minorities from the STEM fields using the BPS:04/09 data. Black STEM students are more 
likely to switch out of STEM fields and to leave school two years post initial enrollment than 
are white STEM students. Regression studies of academic engagement as a function of 
ethnicity show that while there is no main effect for ethnicity, Black students in the 
science/math STEM subfields are significantly more likely to have higher academic 
engagement index scores. Yet, no interaction of ethnicity and engagement (either academic or 
social) significantly predicts STEM persistence. 

Yes, the leaky pipeline exists and is evident in the BPS:04/09 data. Black students who start 
in STEM leave both STEM and school more often than do White students. While this is not 
inconsistent with the leaky pipeline hypothesis and previous research, the continued 
demonstration of this unfortunate fact in this current analysis using recent data indicates that 
STEM retention for underrepresented minorities is still an issue requiring further research. 
Griffith (2010) was using data that was 11 to 22 years old when she asserted that women and 
minorities were even less likely to persist in STEM. Using the most current longitudinal data 
of the BPS:04/09, the condition for women is quite different. Female STEM persistence is no 
longer a critical problem, but the pathways into postsecondary STEM are now more suspect. 
The persistence of underrepresented minorities is crucial if diversity of STEM postsecondary 
graduates is still the goal. The need for more STEM talent with postsecondary education is a 
pressing issue. Using new metrics, the K-12 pipeline into postsecondary STEM education is 
still “too shallow to meet demand” (Alphonse, 2014). The present study shows that 
postsecondary STEM attrition is also an area requiring future research. 

Yes, underrepresented minorities demonstrate differential academic and social engagement 
scores. Most notably, Black science/math students have higher academic engagement scores 
than do either Black engineering students or Latino students in either the science/math or 
engineering/technology STEM subfields. But in answer to the final question, “Do differing 
engagement behaviors contribute to STEM attrition of underrepresented minorities?”— the 
BPS:04/09 data say no. Neither increased nor decreased academic nor social engagement 
improves the likelihood of underrepresented minority students in persisting in the STEM 
fields. 

While neither academic nor social engagement significantly impacts STEM persistence, the 
analyses challenge some of the previous findings. Using the BPS:04/09 gives a nationally 
representative longitudinal view of STEM persistence. The analysis of the BPS:04/09 
including academic and social engagement in the multinomial regression comparing STEM 
stayers, STEM switchers, and college non-persisters (see Table 11) challenges the assertion of 
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Ramirez et al. (2005), and Ohland et al. (2008) of retention as a “campus-based” 
phenomenon. While institution size and selectivity were significantly predictive of staying in 
STEM versus leaving STEM or school altogether, student-level variables have larger effect 
sizes and significance on STEM persistence. This also challenges Daemphle’s (2004) 
statement that switchers were the same “kinds of people” as were non-switchers. The 
individual student characteristics are present before postsecondary enrollment and are more 
powerful predictors of STEM retention than postsecondary engagement. Only when 
researchers look at institution-level student populations could they say they are the “same 
kinds of people”. The varied backgrounds predict STEM retention, specifically, high school 
grade point average, family income, and financial aid received. In combination, examination 
of the national data indicates that there is a selection bias in how students choose their STEM 
institutions. The circumstances seen by researchers studying STEM retention utilizing 
STEM-enrolled populations without controlling for earlier student-level characteristics 
decreased the impact of these earlier experiences. Olhand et al. (2008) hinted at this when 
they stated that engineering majors differed “most notably by a dearth of female students”. 
The current analysis finds that STEM males significantly more often leave school altogether 
compared with females; while fewer in number, the female STEM starters are prone to stay in 
STEM. In effect, females are successfully pre-selecting into STEM albeit at much lower 
numbers. 

The fact that the BPS:04/09 supports the existing literature that Black students who start in 
STEM both leave STEM and postsecondary education at higher rates is both tragic and 
discouraging. In concert with Ohland et al.’s statement that engineering students are 
demographically similar to other college students, it is especially discouraging that Black 
attrition out of engineering is so high relative to White students. 

Engagement in this analysis was measured by student self-reported behaviors. Engagement 
was analyzed as neither a student-level nor an institutional-level factor in this analysis since it 
is functionally a factor that is influenced by both the student and the institution/faculty. 
Students cannot engage with faculty if faculty are not available. Also, faculty involvement 
with clubs and organizations may facilitate both students’ academic and social engagement. 

7. Limitations 

This study explores STEM degree persistence and the impact of both academic and social 
engagement upon persistence, differentiated by ethnicity. This study is dependent upon two 
elements of the BPS:04/09, (1) that the academic and social engagement index scores are 
valid measures of academic and social engagement and (2) that the respondents were accurate 
and reliable in reporting. As is true of all survey data, it has limitations. 

When NCES chose the items composite to the academic engagement index scores, they 
approximated Tinto’s theory choosing two faculty interaction items, an advisor item, and a 
study group item. When they chose the social engagement items, they attempted to select 
campus activities that were not related to the curriculum but related to the campus social 
context. Without understanding the deeper nuances of each campus, this could have blurred 
the academic and social nature of the items. If a campus has a requirement that all clubs have 
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faculty advisors, faculty contact is increased in the measure that attempts to capture a social 
concept. A student may join a club with a faculty advisor to impact academic goals. Likewise, 
meeting with faculty was specifically stated as “informally or socially” in the second faculty 
contact item of the academic engagement index score. The “socially” would imply, to strictly 
follow Tinto’s framework, a social engagement item, yet NCES included this in their 
academic index score. The creation of the indices is only an approximation of student 
academic or social engagement behaviors. Although these are serious limitations of the 
engagement index scores in the BPS:04/09, there is still a difference between a student who 
chooses to engage in academic and social behaviors queried in the BPS:04/09 and a student 
who does not engage academically or socially. Previous studies have found that the indices 
have predictive value for both persistence and degree attainment (Flynn, 2013). 

Another considerable limitation is that NCES chose categorical variables to measure the 
behavioral frequencies of the components for both the academic and social engagement index 
scores in the BPS:04/09. Respondents could choose never, sometimes, or often (coded 0, 1, or 
2 respectively). What one respondent considered as “sometimes” was not necessarily the 
same as what another respondent considered “sometimes”. The creation of the index by 
averaging the four academic and three social engagement index scores was arbitrary at best. 
No examples or stated definitions of “sometimes” and “often” were ever presented to the 
respondents (S. Chrissey, NCES, personal communication, January 9, 2014). A student who is 
conservative in rating their behavioral frequencies would appear less engaged. This is 
especially significant in this study of differences by ethnicity in light of Michelson’s 
“attitude-achievement paradox” (1990). Black high school students were reported to inflate 
their achievement and attitude when self-reporting. The BPS:04/09 did not ask students to 
rate achievement, or attitude; it asked students to report frequency of behavior. While 
Downey, Ainsworth, and Qian (2009) challenge the attitude-achievement paradox specifically 
in large data analysis, the current analysis demonstrates that Black students both switch and 
drop out of STEM more often, while having higher academic engagement scores, 
significantly so for Black science/math students. If the Black respondents to the BPS:04/09 
were inflating their frequencies of the behaviors in the engagement index scores relative to 
the other respondents, the relative impact of the engagement scales can be attributed to the 
paradox rather than to engagement. Future studies with alternate measures of both academic 
and social engagement that are not self-reported (and currently not available in the BPS:04/09 
data) would be necessary to determine whether the Black attitude-achievement paradox is 
impacting behavioral frequency rating in the BPS:04/09. 

Another concern with the BPS’s academic engagement measures is that students can engage 
in all four of the behaviors (meeting with faculty, meeting with faculty informally, meeting 
with advisors, and study groups) for either positive/proactive or reactive/negative impetuses. 
Tinto envisioned academic engagement as curricular in nature and did not differentiate 
between meeting with faculty to further elaborate or improve their academic understanding 
from meeting in reaction to receipt of a poor grade on a paper or exam. In either case, the 
ability and willingness of a student to approach faculty, or advisors, or study groups was 
prophylactic of postsecondary attrition. This study examined staying in STEM. The desire for 
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ethnic diversity in STEM is the societal goal. The need for individual and personal success 
for a student in any field will be the goal of any faculty member or advisor. Thus, increased 
faculty contact and/or advisor interaction may result in a change of field that is appropriate 
for the student (albeit counter to the societal goal of increasing STEM persistence and ethnic 
diversity). The fundamental limitation of this study is that it assumes staying in STEM meets 
a societal goal but negates the possibility that leaving STEM could be meeting a personal 
goal. The data indicate that more Black students leave STEM and Black science/math starters 
also demonstrate higher academic engagement. The behaviors measured by the academic 
engagement measures (and the concepts theorized by Tinto) were not designed to retain 
students in an initial field. They were designed to explore postsecondary persistence and 
degree attainment. Different questions and measures are necessary to address what keeps 
underrepresent minority students in STEM, quite possibly not using the BPS:04/09. 

Finally, when the BPS:04/09 was developed (well before 2003 when it was initially 
implemented), both academic and social engagement was more personal and less 
technologically augmented. Even though the BPS:04/09 queries faculty contact “including 
email”, the advancements in virtual interactions have changed the educational landscape. 
Message boards, learning management systems, and even social media have changed how 
students interact with faculty and fellow students. The ease of email communication, online 
discussion, and even social networking may profoundly alter how students currently engage 
with faculty. These technological changes were not considered in the creation of the 
BPS:04/09 and therefore the BPS:04/09 relies on student self-reported behaviors that are 
indicative of engagement. Technology may impact current and future conceptualizations of 
academic integration and engagement. 
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Notes 

Note 1. In the sample of all majors (n = 8700), six hundred forty students (7% of the sample) 
have missing tuition data and the mean tuition value of $11,030 was substituted for the 
missing values for those cases. Four hundred twenty subjects (5% of the sample) have 
missing data on residential status and the mean dummy value of “0,” indicating “residential 
school”, is used for students missing values for the non-residential dummy variable. Four 
hundred twenty students (5% of the sample) have missing data on the three school size 
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dummy variables accounting for large, medium, and small institution size. These missing 
values are all assigned the “1” within the Targe school population’ dummy variable. 
Comparison of listwise deletion results to results using mean substitution demonstrated no 
significant change in coefficients for any of the engagement variables. The mean substitution 
method yielded more conservative results than did the analyses utilizing listwise deletion. 

Note 2. Approximately of 110 (8% of the STEM starter sample) students indicated two racial 
categories within the BPS:04/09 dataset. Fifty participants (3%) indicated both Latino and 
Asian. Less than ten participants (< 1%) indicated their as both Latino and White, as both 
black and white, both, or both Asian and Black. NCES/US Department of Education 
reporting policies reclassified students who reported both Black and Latino as Latino. This 
change in the way in which NCES reclassified multiracial Latino students was short-lived but 
impacted the BPS:04/09 (Newman, 2014) Two separate fully modeled analyses addressed an 
additional multiracial ethnic category and demonstrated no significant change upon either the 
engagement migration analyses. 


Glossary 

BPS: Beginning Postsecondary Longitudinal Student Survey; 

BPS:04/09: Beginning Postsecondary Longitudinal Student Survey 2004-2009; 

NCES: The National Center for Education Statistics’; 

OLS: Ordinary Least Squared; 

RRR: Relative Risk Ratio; 

STEM: Science, Technology, Engineering, and Math. More specifically, the academic fiedls 
of study that address Science, Engineering, Engineering and Math. 
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